Influence of weight-decay training in input selection methods

We describe the results of a research on the effect of weight-decay (WD) in input selection methods based on the analysis of a trained multilayer feedforward network. It was proposed by some authors to train the network with WD before applying this type of methods. The influence of WD in sixteen different input selection methods is empirically analyzed with a total of seven classification problems. We show that the performance variation of the input selection methods by introducing WD depends on the particular method. But for some of them, the use of WD can deteriorate their efficiency. Furthermore, it seems that WD improves the efficiency of the worst methods and deteriorates the performance of the best ones. In that sense, it diminishes the differences among different methods. We think that the use of weight-decay with this type of input selection methods should be avoided because the results are not good and also the use of weight-decay supposes a complication of the procedure.